Input output stability of recurrent neural networks

i Foreword Recurrent neural networks are an attractive tool for both practical applications and for the modeling of biological nerve nets, but their successful application requires an understanding of their dynamical properties, in particular, their stability. The present work provides an in-depth study of this challenging issue and contributes a number of new results that are also important for a broader class of recurrent systems containing nonlinear and even time-delayed feedback. The approach is based on modern concepts of control theory, with an emphasis on techniques that have been developed for the analysis of feedback systems with parameter uncertainties during recent years. In addition to the analytic derivations, the author demonstrates how the derived criteria can be numerically evaluated with modern techniques for quadratic optimization. Some of the techniques are then illustrated for the example of using a fully recurrent network for learning the dynamics of a chaotic system. The present monograph will offer the mathematically inclined reader an unusual, but powerful approach to the stability analysis of recurrent systems and acquaint him with many advanced concepts that he may find useful for his own research. Acknowledgement The presented work was mainly carried out in the neural network group of the Depart-Dynamik rekurrenter neuronaler Netze ". First of all I would like to thank Helge, who's scientific enthusiasm attracted me to the group a number of years ago and who's optimism always remained to be a powerful inspiration throughout my work. Without his personal confidence and scientific support this work would not have been settled. A second root of the presented work lies in Russia. This is true regarding content, because a lot of techniques I use in this thesis were first considered there, but it is even more the case for me personally. Supported by a DAAD grant, in 1995/96 I had the opportunity to spend one year at the former Electrotechnical Institute in St. Petersburg, which is situated in – No. 5, Professor Popov street ! There I met Prof. Dr. I. B. Junger, who first acquainted me with frequency and input-output methods. Without his help and the support from Dr. Oleg Gerasimov I could not have reached the level of understanding in " frequency theory " – as they call it – which was necessary for the current work. Since the year 1995 this connection never was cut and recently even reached a new level, because Prof. …